{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "077b0f04",
   "metadata": {
    "origin_pos": 0
   },
   "source": [
    "# 机器翻译与数据集\n",
    ":label:`sec_machine_translation`\n",
    "\n",
    "语言模型是自然语言处理的关键，\n",
    "而*机器翻译*是语言模型最成功的基准测试。\n",
    "因为机器翻译正是将输入序列转换成输出序列的\n",
    "*序列转换模型*（sequence transduction）的核心问题。\n",
    "序列转换模型在各类现代人工智能应用中发挥着至关重要的作用，\n",
    "因此我们将其做为本章剩余部分和 :numref:`chap_attention`的重点。\n",
    "为此，本节将介绍机器翻译问题及其后文需要使用的数据集。\n",
    "\n",
    "*机器翻译*（machine translation）指的是\n",
    "将序列从一种语言自动翻译成另一种语言。\n",
    "事实上，这个研究领域可以追溯到数字计算机发明后不久的20世纪40年代，\n",
    "特别是在第二次世界大战中使用计算机破解语言编码。\n",
    "几十年来，在使用神经网络进行端到端学习的兴起之前，\n",
    "统计学方法在这一领域一直占据主导地位\n",
    " :cite:`Brown.Cocke.Della-Pietra.ea.1988,Brown.Cocke.Della-Pietra.ea.1990`。\n",
    "因为*统计机器翻译*（statistical machine translation）涉及了\n",
    "翻译模型和语言模型等组成部分的统计分析，\n",
    "因此基于神经网络的方法通常被称为\n",
    "*神经机器翻译*（neural machine translation），\n",
    "用于将两种翻译模型区分开来。\n",
    "\n",
    "本书的关注点是神经网络机器翻译方法，强调的是端到端的学习。\n",
    "与 :numref:`sec_language_model`中的语料库\n",
    "是单一语言的语言模型问题存在不同，\n",
    "机器翻译的数据集是由源语言和目标语言的文本序列对组成的。\n",
    "因此，我们需要一种完全不同的方法来预处理机器翻译数据集，\n",
    "而不是复用语言模型的预处理程序。\n",
    "下面，我们看一下如何将预处理后的数据加载到小批量中用于训练。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "38f128f5",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-08-18T07:07:28.568184Z",
     "iopub.status.busy": "2023-08-18T07:07:28.567577Z",
     "iopub.status.idle": "2023-08-18T07:07:30.535405Z",
     "shell.execute_reply": "2023-08-18T07:07:30.534582Z"
    },
    "origin_pos": 2,
    "tab": [
     "pytorch"
    ]
   },
   "outputs": [],
   "source": [
    "import os\n",
    "import torch\n",
    "from d2l import torch as d2l"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4f0458ec",
   "metadata": {
    "origin_pos": 5
   },
   "source": [
    "## [**下载和预处理数据集**]\n",
    "\n",
    "首先，下载一个由[Tatoeba项目的双语句子对](http://www.manythings.org/anki/)\n",
    "组成的“英－法”数据集，数据集中的每一行都是制表符分隔的文本序列对，\n",
    "序列对由英文文本序列和翻译后的法语文本序列组成。\n",
    "请注意，每个文本序列可以是一个句子，\n",
    "也可以是包含多个句子的一个段落。\n",
    "在这个将英语翻译成法语的机器翻译问题中，\n",
    "英语是*源语言*（source language），\n",
    "法语是*目标语言*（target language）。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "b3461d76",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-08-18T07:07:30.539676Z",
     "iopub.status.busy": "2023-08-18T07:07:30.539042Z",
     "iopub.status.idle": "2023-08-18T07:07:30.809623Z",
     "shell.execute_reply": "2023-08-18T07:07:30.808727Z"
    },
    "origin_pos": 6,
    "tab": [
     "pytorch"
    ]
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading ../data/fra-eng.zip from http://d2l-data.s3-accelerate.amazonaws.com/fra-eng.zip...\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Go.\tVa !\n",
      "Hi.\tSalut !\n",
      "Run!\tCours !\n",
      "Run!\tCourez !\n",
      "Who?\tQui ?\n",
      "Wow!\tÇa alors !\n",
      "\n"
     ]
    }
   ],
   "source": [
    "#@save\n",
    "d2l.DATA_HUB['fra-eng'] = (d2l.DATA_URL + 'fra-eng.zip',\n",
    "                           '94646ad1522d915e7b0f9296181140edcf86a4f5')\n",
    "\n",
    "#@save\n",
    "def read_data_nmt():\n",
    "    \"\"\"载入“英语－法语”数据集\"\"\"\n",
    "    data_dir = d2l.download_extract('fra-eng')\n",
    "    with open(os.path.join(data_dir, 'fra.txt'), 'r',\n",
    "             encoding='utf-8') as f:\n",
    "        return f.read()\n",
    "\n",
    "raw_text = read_data_nmt()\n",
    "print(raw_text[:75])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "de8c081f",
   "metadata": {
    "origin_pos": 7
   },
   "source": [
    "下载数据集后，原始文本数据需要经过[**几个预处理步骤**]。\n",
    "例如，我们用空格代替*不间断空格*（non-breaking space），\n",
    "使用小写字母替换大写字母，并在单词和标点符号之间插入空格。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "114c461d",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-08-18T07:07:30.813835Z",
     "iopub.status.busy": "2023-08-18T07:07:30.813273Z",
     "iopub.status.idle": "2023-08-18T07:07:36.581927Z",
     "shell.execute_reply": "2023-08-18T07:07:36.580959Z"
    },
    "origin_pos": 8,
    "tab": [
     "pytorch"
    ]
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "go .\tva !\n",
      "hi .\tsalut !\n",
      "run !\tcours !\n",
      "run !\tcourez !\n",
      "who ?\tqui ?\n",
      "wow !\tça alors !\n"
     ]
    }
   ],
   "source": [
    "#@save\n",
    "def preprocess_nmt(text):\n",
    "    \"\"\"预处理“英语－法语”数据集\"\"\"\n",
    "    def no_space(char, prev_char):\n",
    "        return char in set(',.!?') and prev_char != ' '\n",
    "\n",
    "    # 使用空格替换不间断空格\n",
    "    # 使用小写字母替换大写字母\n",
    "    text = text.replace('\\u202f', ' ').replace('\\xa0', ' ').lower()\n",
    "    # 在单词和标点符号之间插入空格\n",
    "    out = [' ' + char if i > 0 and no_space(char, text[i - 1]) else char\n",
    "           for i, char in enumerate(text)]\n",
    "    return ''.join(out)\n",
    "\n",
    "text = preprocess_nmt(raw_text)\n",
    "print(text[:80])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e4048187",
   "metadata": {
    "origin_pos": 9
   },
   "source": [
    "## [**词元化**]\n",
    "\n",
    "与 :numref:`sec_language_model`中的字符级词元化不同，\n",
    "在机器翻译中，我们更喜欢单词级词元化\n",
    "（最先进的模型可能使用更高级的词元化技术）。\n",
    "下面的`tokenize_nmt`函数对前`num_examples`个文本序列对进行词元，\n",
    "其中每个词元要么是一个词，要么是一个标点符号。\n",
    "此函数返回两个词元列表：`source`和`target`：\n",
    "`source[i]`是源语言（这里是英语）第$i$个文本序列的词元列表，\n",
    "`target[i]`是目标语言（这里是法语）第$i$个文本序列的词元列表。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "cc08d1a5",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-08-18T07:07:36.585962Z",
     "iopub.status.busy": "2023-08-18T07:07:36.585396Z",
     "iopub.status.idle": "2023-08-18T07:07:37.431130Z",
     "shell.execute_reply": "2023-08-18T07:07:37.430360Z"
    },
    "origin_pos": 10,
    "tab": [
     "pytorch"
    ]
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "([['go', '.'],\n",
       "  ['hi', '.'],\n",
       "  ['run', '!'],\n",
       "  ['run', '!'],\n",
       "  ['who', '?'],\n",
       "  ['wow', '!']],\n",
       " [['va', '!'],\n",
       "  ['salut', '!'],\n",
       "  ['cours', '!'],\n",
       "  ['courez', '!'],\n",
       "  ['qui', '?'],\n",
       "  ['ça', 'alors', '!']])"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#@save\n",
    "def tokenize_nmt(text, num_examples=None):\n",
    "    \"\"\"词元化“英语－法语”数据数据集\"\"\"\n",
    "    source, target = [], []\n",
    "    for i, line in enumerate(text.split('\\n')):\n",
    "        if num_examples and i > num_examples:\n",
    "            break\n",
    "        parts = line.split('\\t')\n",
    "        if len(parts) == 2:\n",
    "            source.append(parts[0].split(' '))\n",
    "            target.append(parts[1].split(' '))\n",
    "    return source, target\n",
    "\n",
    "source, target = tokenize_nmt(text)\n",
    "source[:6], target[:6]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1d8ecec6",
   "metadata": {
    "origin_pos": 11
   },
   "source": [
    "让我们[**绘制每个文本序列所包含的词元数量的直方图**]。\n",
    "在这个简单的“英－法”数据集中，大多数文本序列的词元数量少于$20$个。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "118a08f5",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-08-18T07:07:37.436349Z",
     "iopub.status.busy": "2023-08-18T07:07:37.435794Z",
     "iopub.status.idle": "2023-08-18T07:07:37.726273Z",
     "shell.execute_reply": "2023-08-18T07:07:37.725467Z"
    },
    "origin_pos": 12,
    "tab": [
     "pytorch"
    ]
   },
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       "  \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<svg xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"274.320356pt\" height=\"180.65625pt\" viewBox=\"0 0 274.320356 180.65625\" xmlns=\"http://www.w3.org/2000/svg\" version=\"1.1\">\n",
       " <metadata>\n",
       "  <rdf:RDF xmlns:dc=\"http://purl.org/dc/elements/1.1/\" xmlns:cc=\"http://creativecommons.org/ns#\" xmlns:rdf=\"http://www.w3.org/1999/02/22-rdf-syntax-ns#\">\n",
       "   <cc:Work>\n",
       "    <dc:type rdf:resource=\"http://purl.org/dc/dcmitype/StillImage\"/>\n",
       "    <dc:date>2023-08-18T07:07:37.679341</dc:date>\n",
       "    <dc:format>image/svg+xml</dc:format>\n",
       "    <dc:creator>\n",
       "     <cc:Agent>\n",
       "      <dc:title>Matplotlib v3.5.1, https://matplotlib.org/</dc:title>\n",
       "     </cc:Agent>\n",
       "    </dc:creator>\n",
       "   </cc:Work>\n",
       "  </rdf:RDF>\n",
       " </metadata>\n",
       " <defs>\n",
       "  <style type=\"text/css\">*{stroke-linejoin: round; stroke-linecap: butt}</style>\n",
       " </defs>\n",
       " <g id=\"figure_1\">\n",
       "  <g id=\"patch_1\">\n",
       "   <path d=\"M 0 180.65625 \n",
       "L 274.320356 180.65625 \n",
       "L 274.320356 0 \n",
       "L 0 0 \n",
       "L 0 180.65625 \n",
       "z\n",
       "\" style=\"fill: none\"/>\n",
       "  </g>\n",
       "  <g id=\"axes_1\">\n",
       "   <g id=\"patch_2\">\n",
       "    <path d=\"M 66.053125 143.1 \n",
       "L 261.353125 143.1 \n",
       "L 261.353125 7.2 \n",
       "L 66.053125 7.2 \n",
       "z\n",
       "\" style=\"fill: #ffffff\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_3\">\n",
       "    <path d=\"M 74.930398 143.1 \n",
       "L 82.177151 143.1 \n",
       "L 82.177151 13.671429 \n",
       "L 74.930398 13.671429 \n",
       "z\n",
       "\" clip-path=\"url(#p4f217efe07)\" style=\"fill: #1f77b4\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_4\">\n",
       "    <path d=\"M 93.047281 143.1 \n",
       "L 100.294034 143.1 \n",
       "L 100.294034 69.112838 \n",
       "L 93.047281 69.112838 \n",
       "z\n",
       "\" clip-path=\"url(#p4f217efe07)\" style=\"fill: #1f77b4\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_5\">\n",
       "    <path d=\"M 111.164164 143.1 \n",
       "L 118.410917 143.1 \n",
       "L 118.410917 138.560551 \n",
       "L 111.164164 138.560551 \n",
       "z\n",
       "\" clip-path=\"url(#p4f217efe07)\" style=\"fill: #1f77b4\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_6\">\n",
       "    <path d=\"M 129.281047 143.1 \n",
       "L 136.5278 143.1 \n",
       "L 136.5278 142.652168 \n",
       "L 129.281047 142.652168 \n",
       "z\n",
       "\" clip-path=\"url(#p4f217efe07)\" style=\"fill: #1f77b4\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_7\">\n",
       "    <path d=\"M 147.39793 143.1 \n",
       "L 154.644683 143.1 \n",
       "L 154.644683 143.045112 \n",
       "L 147.39793 143.045112 \n",
       "z\n",
       "\" clip-path=\"url(#p4f217efe07)\" style=\"fill: #1f77b4\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_8\">\n",
       "    <path d=\"M 165.514813 143.1 \n",
       "L 172.761567 143.1 \n",
       "L 172.761567 143.083783 \n",
       "L 165.514813 143.083783 \n",
       "z\n",
       "\" clip-path=\"url(#p4f217efe07)\" style=\"fill: #1f77b4\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_9\">\n",
       "    <path d=\"M 183.631696 143.1 \n",
       "L 190.87845 143.1 \n",
       "L 190.87845 143.092515 \n",
       "L 183.631696 143.092515 \n",
       "z\n",
       "\" clip-path=\"url(#p4f217efe07)\" style=\"fill: #1f77b4\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_10\">\n",
       "    <path d=\"M 201.74858 143.1 \n",
       "L 208.995333 143.1 \n",
       "L 208.995333 143.098753 \n",
       "L 201.74858 143.098753 \n",
       "z\n",
       "\" clip-path=\"url(#p4f217efe07)\" style=\"fill: #1f77b4\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_11\">\n",
       "    <path d=\"M 219.865463 143.1 \n",
       "L 227.112216 143.1 \n",
       "L 227.112216 143.097505 \n",
       "L 219.865463 143.097505 \n",
       "z\n",
       "\" clip-path=\"url(#p4f217efe07)\" style=\"fill: #1f77b4\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_12\">\n",
       "    <path d=\"M 237.982346 143.1 \n",
       "L 245.229099 143.1 \n",
       "L 245.229099 143.1 \n",
       "L 237.982346 143.1 \n",
       "z\n",
       "\" clip-path=\"url(#p4f217efe07)\" style=\"fill: #1f77b4\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_13\">\n",
       "    <path d=\"M 82.177151 143.1 \n",
       "L 89.423904 143.1 \n",
       "L 89.423904 26.397854 \n",
       "L 82.177151 26.397854 \n",
       "z\n",
       "\" clip-path=\"url(#p4f217efe07)\" style=\"fill: url(#h446a4b1fc9)\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_14\">\n",
       "    <path d=\"M 100.294034 143.1 \n",
       "L 107.540787 143.1 \n",
       "L 107.540787 59.453878 \n",
       "L 100.294034 59.453878 \n",
       "z\n",
       "\" clip-path=\"url(#p4f217efe07)\" style=\"fill: url(#h446a4b1fc9)\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_15\">\n",
       "    <path d=\"M 118.410917 143.1 \n",
       "L 125.65767 143.1 \n",
       "L 125.65767 136.044456 \n",
       "L 118.410917 136.044456 \n",
       "z\n",
       "\" clip-path=\"url(#p4f217efe07)\" style=\"fill: url(#h446a4b1fc9)\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_16\">\n",
       "    <path d=\"M 136.5278 143.1 \n",
       "L 143.774554 143.1 \n",
       "L 143.774554 142.176891 \n",
       "L 136.5278 142.176891 \n",
       "z\n",
       "\" clip-path=\"url(#p4f217efe07)\" style=\"fill: url(#h446a4b1fc9)\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_17\">\n",
       "    <path d=\"M 154.644683 143.1 \n",
       "L 161.891437 143.1 \n",
       "L 161.891437 142.993967 \n",
       "L 154.644683 142.993967 \n",
       "z\n",
       "\" clip-path=\"url(#p4f217efe07)\" style=\"fill: url(#h446a4b1fc9)\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_18\">\n",
       "    <path d=\"M 172.761567 143.1 \n",
       "L 180.00832 143.1 \n",
       "L 180.00832 143.058834 \n",
       "L 172.761567 143.058834 \n",
       "z\n",
       "\" clip-path=\"url(#p4f217efe07)\" style=\"fill: url(#h446a4b1fc9)\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_19\">\n",
       "    <path d=\"M 190.87845 143.1 \n",
       "L 198.125203 143.1 \n",
       "L 198.125203 143.097505 \n",
       "L 190.87845 143.097505 \n",
       "z\n",
       "\" clip-path=\"url(#p4f217efe07)\" style=\"fill: url(#h446a4b1fc9)\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_20\">\n",
       "    <path d=\"M 208.995333 143.1 \n",
       "L 216.242086 143.1 \n",
       "L 216.242086 143.097505 \n",
       "L 208.995333 143.097505 \n",
       "z\n",
       "\" clip-path=\"url(#p4f217efe07)\" style=\"fill: url(#h446a4b1fc9)\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_21\">\n",
       "    <path d=\"M 227.112216 143.1 \n",
       "L 234.358969 143.1 \n",
       "L 234.358969 143.097505 \n",
       "L 227.112216 143.097505 \n",
       "z\n",
       "\" clip-path=\"url(#p4f217efe07)\" style=\"fill: url(#h446a4b1fc9)\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_22\">\n",
       "    <path d=\"M 245.229099 143.1 \n",
       "L 252.475852 143.1 \n",
       "L 252.475852 143.096258 \n",
       "L 245.229099 143.096258 \n",
       "z\n",
       "\" clip-path=\"url(#p4f217efe07)\" style=\"fill: url(#h446a4b1fc9)\"/>\n",
       "   </g>\n",
       "   <g id=\"matplotlib.axis_1\">\n",
       "    <g id=\"xtick_1\">\n",
       "     <g id=\"line2d_1\">\n",
       "      <defs>\n",
       "       <path id=\"mfde7d55445\" d=\"M 0 0 \n",
       "L 0 3.5 \n",
       "\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use xlink:href=\"#mfde7d55445\" x=\"66.648394\" y=\"143.1\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_1\">\n",
       "      <!-- 0 -->\n",
       "      <g transform=\"translate(63.467144 157.698438)scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-30\" d=\"M 2034 4250 \n",
       "Q 1547 4250 1301 3770 \n",
       "Q 1056 3291 1056 2328 \n",
       "Q 1056 1369 1301 889 \n",
       "Q 1547 409 2034 409 \n",
       "Q 2525 409 2770 889 \n",
       "Q 3016 1369 3016 2328 \n",
       "Q 3016 3291 2770 3770 \n",
       "Q 2525 4250 2034 4250 \n",
       "z\n",
       "M 2034 4750 \n",
       "Q 2819 4750 3233 4129 \n",
       "Q 3647 3509 3647 2328 \n",
       "Q 3647 1150 3233 529 \n",
       "Q 2819 -91 2034 -91 \n",
       "Q 1250 -91 836 529 \n",
       "Q 422 1150 422 2328 \n",
       "Q 422 3509 836 4129 \n",
       "Q 1250 4750 2034 4750 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_2\">\n",
       "     <g id=\"line2d_2\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#mfde7d55445\" x=\"131.351548\" y=\"143.1\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_2\">\n",
       "      <!-- 20 -->\n",
       "      <g transform=\"translate(124.989048 157.698438)scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-32\" d=\"M 1228 531 \n",
       "L 3431 531 \n",
       "L 3431 0 \n",
       "L 469 0 \n",
       "L 469 531 \n",
       "Q 828 903 1448 1529 \n",
       "Q 2069 2156 2228 2338 \n",
       "Q 2531 2678 2651 2914 \n",
       "Q 2772 3150 2772 3378 \n",
       "Q 2772 3750 2511 3984 \n",
       "Q 2250 4219 1831 4219 \n",
       "Q 1534 4219 1204 4116 \n",
       "Q 875 4013 500 3803 \n",
       "L 500 4441 \n",
       "Q 881 4594 1212 4672 \n",
       "Q 1544 4750 1819 4750 \n",
       "Q 2544 4750 2975 4387 \n",
       "Q 3406 4025 3406 3419 \n",
       "Q 3406 3131 3298 2873 \n",
       "Q 3191 2616 2906 2266 \n",
       "Q 2828 2175 2409 1742 \n",
       "Q 1991 1309 1228 531 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-32\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-30\" x=\"63.623047\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_3\">\n",
       "     <g id=\"line2d_3\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#mfde7d55445\" x=\"196.054702\" y=\"143.1\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_3\">\n",
       "      <!-- 40 -->\n",
       "      <g transform=\"translate(189.692202 157.698438)scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-34\" d=\"M 2419 4116 \n",
       "L 825 1625 \n",
       "L 2419 1625 \n",
       "L 2419 4116 \n",
       "z\n",
       "M 2253 4666 \n",
       "L 3047 4666 \n",
       "L 3047 1625 \n",
       "L 3713 1625 \n",
       "L 3713 1100 \n",
       "L 3047 1100 \n",
       "L 3047 0 \n",
       "L 2419 0 \n",
       "L 2419 1100 \n",
       "L 313 1100 \n",
       "L 313 1709 \n",
       "L 2253 4666 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-34\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-30\" x=\"63.623047\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_4\">\n",
       "     <g id=\"line2d_4\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#mfde7d55445\" x=\"260.757856\" y=\"143.1\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_4\">\n",
       "      <!-- 60 -->\n",
       "      <g transform=\"translate(254.395356 157.698438)scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-36\" d=\"M 2113 2584 \n",
       "Q 1688 2584 1439 2293 \n",
       "Q 1191 2003 1191 1497 \n",
       "Q 1191 994 1439 701 \n",
       "Q 1688 409 2113 409 \n",
       "Q 2538 409 2786 701 \n",
       "Q 3034 994 3034 1497 \n",
       "Q 3034 2003 2786 2293 \n",
       "Q 2538 2584 2113 2584 \n",
       "z\n",
       "M 3366 4563 \n",
       "L 3366 3988 \n",
       "Q 3128 4100 2886 4159 \n",
       "Q 2644 4219 2406 4219 \n",
       "Q 1781 4219 1451 3797 \n",
       "Q 1122 3375 1075 2522 \n",
       "Q 1259 2794 1537 2939 \n",
       "Q 1816 3084 2150 3084 \n",
       "Q 2853 3084 3261 2657 \n",
       "Q 3669 2231 3669 1497 \n",
       "Q 3669 778 3244 343 \n",
       "Q 2819 -91 2113 -91 \n",
       "Q 1303 -91 875 529 \n",
       "Q 447 1150 447 2328 \n",
       "Q 447 3434 972 4092 \n",
       "Q 1497 4750 2381 4750 \n",
       "Q 2619 4750 2861 4703 \n",
       "Q 3103 4656 3366 4563 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-36\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-30\" x=\"63.623047\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"text_5\">\n",
       "     <!-- # tokens per sequence -->\n",
       "     <g transform=\"translate(105.760937 171.376563)scale(0.1 -0.1)\">\n",
       "      <defs>\n",
       "       <path id=\"DejaVuSans-23\" d=\"M 3272 2816 \n",
       "L 2363 2816 \n",
       "L 2100 1772 \n",
       "L 3016 1772 \n",
       "L 3272 2816 \n",
       "z\n",
       "M 2803 4594 \n",
       "L 2478 3297 \n",
       "L 3391 3297 \n",
       "L 3719 4594 \n",
       "L 4219 4594 \n",
       "L 3897 3297 \n",
       "L 4872 3297 \n",
       "L 4872 2816 \n",
       "L 3775 2816 \n",
       "L 3519 1772 \n",
       "L 4513 1772 \n",
       "L 4513 1294 \n",
       "L 3397 1294 \n",
       "L 3072 0 \n",
       "L 2572 0 \n",
       "L 2894 1294 \n",
       "L 1978 1294 \n",
       "L 1656 0 \n",
       "L 1153 0 \n",
       "L 1478 1294 \n",
       "L 494 1294 \n",
       "L 494 1772 \n",
       "L 1594 1772 \n",
       "L 1856 2816 \n",
       "L 850 2816 \n",
       "L 850 3297 \n",
       "L 1978 3297 \n",
       "L 2297 4594 \n",
       "L 2803 4594 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-20\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-74\" d=\"M 1172 4494 \n",
       "L 1172 3500 \n",
       "L 2356 3500 \n",
       "L 2356 3053 \n",
       "L 1172 3053 \n",
       "L 1172 1153 \n",
       "Q 1172 725 1289 603 \n",
       "Q 1406 481 1766 481 \n",
       "L 2356 481 \n",
       "L 2356 0 \n",
       "L 1766 0 \n",
       "Q 1100 0 847 248 \n",
       "Q 594 497 594 1153 \n",
       "L 594 3053 \n",
       "L 172 3053 \n",
       "L 172 3500 \n",
       "L 594 3500 \n",
       "L 594 4494 \n",
       "L 1172 4494 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-6f\" d=\"M 1959 3097 \n",
       "Q 1497 3097 1228 2736 \n",
       "Q 959 2375 959 1747 \n",
       "Q 959 1119 1226 758 \n",
       "Q 1494 397 1959 397 \n",
       "Q 2419 397 2687 759 \n",
       "Q 2956 1122 2956 1747 \n",
       "Q 2956 2369 2687 2733 \n",
       "Q 2419 3097 1959 3097 \n",
       "z\n",
       "M 1959 3584 \n",
       "Q 2709 3584 3137 3096 \n",
       "Q 3566 2609 3566 1747 \n",
       "Q 3566 888 3137 398 \n",
       "Q 2709 -91 1959 -91 \n",
       "Q 1206 -91 779 398 \n",
       "Q 353 888 353 1747 \n",
       "Q 353 2609 779 3096 \n",
       "Q 1206 3584 1959 3584 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-6b\" d=\"M 581 4863 \n",
       "L 1159 4863 \n",
       "L 1159 1991 \n",
       "L 2875 3500 \n",
       "L 3609 3500 \n",
       "L 1753 1863 \n",
       "L 3688 0 \n",
       "L 2938 0 \n",
       "L 1159 1709 \n",
       "L 1159 0 \n",
       "L 581 0 \n",
       "L 581 4863 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-65\" d=\"M 3597 1894 \n",
       "L 3597 1613 \n",
       "L 953 1613 \n",
       "Q 991 1019 1311 708 \n",
       "Q 1631 397 2203 397 \n",
       "Q 2534 397 2845 478 \n",
       "Q 3156 559 3463 722 \n",
       "L 3463 178 \n",
       "Q 3153 47 2828 -22 \n",
       "Q 2503 -91 2169 -91 \n",
       "Q 1331 -91 842 396 \n",
       "Q 353 884 353 1716 \n",
       "Q 353 2575 817 3079 \n",
       "Q 1281 3584 2069 3584 \n",
       "Q 2775 3584 3186 3129 \n",
       "Q 3597 2675 3597 1894 \n",
       "z\n",
       "M 3022 2063 \n",
       "Q 3016 2534 2758 2815 \n",
       "Q 2500 3097 2075 3097 \n",
       "Q 1594 3097 1305 2825 \n",
       "Q 1016 2553 972 2059 \n",
       "L 3022 2063 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-6e\" d=\"M 3513 2113 \n",
       "L 3513 0 \n",
       "L 2938 0 \n",
       "L 2938 2094 \n",
       "Q 2938 2591 2744 2837 \n",
       "Q 2550 3084 2163 3084 \n",
       "Q 1697 3084 1428 2787 \n",
       "Q 1159 2491 1159 1978 \n",
       "L 1159 0 \n",
       "L 581 0 \n",
       "L 581 3500 \n",
       "L 1159 3500 \n",
       "L 1159 2956 \n",
       "Q 1366 3272 1645 3428 \n",
       "Q 1925 3584 2291 3584 \n",
       "Q 2894 3584 3203 3211 \n",
       "Q 3513 2838 3513 2113 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-73\" d=\"M 2834 3397 \n",
       "L 2834 2853 \n",
       "Q 2591 2978 2328 3040 \n",
       "Q 2066 3103 1784 3103 \n",
       "Q 1356 3103 1142 2972 \n",
       "Q 928 2841 928 2578 \n",
       "Q 928 2378 1081 2264 \n",
       "Q 1234 2150 1697 2047 \n",
       "L 1894 2003 \n",
       "Q 2506 1872 2764 1633 \n",
       "Q 3022 1394 3022 966 \n",
       "Q 3022 478 2636 193 \n",
       "Q 2250 -91 1575 -91 \n",
       "Q 1294 -91 989 -36 \n",
       "Q 684 19 347 128 \n",
       "L 347 722 \n",
       "Q 666 556 975 473 \n",
       "Q 1284 391 1588 391 \n",
       "Q 1994 391 2212 530 \n",
       "Q 2431 669 2431 922 \n",
       "Q 2431 1156 2273 1281 \n",
       "Q 2116 1406 1581 1522 \n",
       "L 1381 1569 \n",
       "Q 847 1681 609 1914 \n",
       "Q 372 2147 372 2553 \n",
       "Q 372 3047 722 3315 \n",
       "Q 1072 3584 1716 3584 \n",
       "Q 2034 3584 2315 3537 \n",
       "Q 2597 3491 2834 3397 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-70\" d=\"M 1159 525 \n",
       "L 1159 -1331 \n",
       "L 581 -1331 \n",
       "L 581 3500 \n",
       "L 1159 3500 \n",
       "L 1159 2969 \n",
       "Q 1341 3281 1617 3432 \n",
       "Q 1894 3584 2278 3584 \n",
       "Q 2916 3584 3314 3078 \n",
       "Q 3713 2572 3713 1747 \n",
       "Q 3713 922 3314 415 \n",
       "Q 2916 -91 2278 -91 \n",
       "Q 1894 -91 1617 61 \n",
       "Q 1341 213 1159 525 \n",
       "z\n",
       "M 3116 1747 \n",
       "Q 3116 2381 2855 2742 \n",
       "Q 2594 3103 2138 3103 \n",
       "Q 1681 3103 1420 2742 \n",
       "Q 1159 2381 1159 1747 \n",
       "Q 1159 1113 1420 752 \n",
       "Q 1681 391 2138 391 \n",
       "Q 2594 391 2855 752 \n",
       "Q 3116 1113 3116 1747 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-72\" d=\"M 2631 2963 \n",
       "Q 2534 3019 2420 3045 \n",
       "Q 2306 3072 2169 3072 \n",
       "Q 1681 3072 1420 2755 \n",
       "Q 1159 2438 1159 1844 \n",
       "L 1159 0 \n",
       "L 581 0 \n",
       "L 581 3500 \n",
       "L 1159 3500 \n",
       "L 1159 2956 \n",
       "Q 1341 3275 1631 3429 \n",
       "Q 1922 3584 2338 3584 \n",
       "Q 2397 3584 2469 3576 \n",
       "Q 2541 3569 2628 3553 \n",
       "L 2631 2963 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-71\" d=\"M 947 1747 \n",
       "Q 947 1113 1208 752 \n",
       "Q 1469 391 1925 391 \n",
       "Q 2381 391 2643 752 \n",
       "Q 2906 1113 2906 1747 \n",
       "Q 2906 2381 2643 2742 \n",
       "Q 2381 3103 1925 3103 \n",
       "Q 1469 3103 1208 2742 \n",
       "Q 947 2381 947 1747 \n",
       "z\n",
       "M 2906 525 \n",
       "Q 2725 213 2448 61 \n",
       "Q 2172 -91 1784 -91 \n",
       "Q 1150 -91 751 415 \n",
       "Q 353 922 353 1747 \n",
       "Q 353 2572 751 3078 \n",
       "Q 1150 3584 1784 3584 \n",
       "Q 2172 3584 2448 3432 \n",
       "Q 2725 3281 2906 2969 \n",
       "L 2906 3500 \n",
       "L 3481 3500 \n",
       "L 3481 -1331 \n",
       "L 2906 -1331 \n",
       "L 2906 525 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-75\" d=\"M 544 1381 \n",
       "L 544 3500 \n",
       "L 1119 3500 \n",
       "L 1119 1403 \n",
       "Q 1119 906 1312 657 \n",
       "Q 1506 409 1894 409 \n",
       "Q 2359 409 2629 706 \n",
       "Q 2900 1003 2900 1516 \n",
       "L 2900 3500 \n",
       "L 3475 3500 \n",
       "L 3475 0 \n",
       "L 2900 0 \n",
       "L 2900 538 \n",
       "Q 2691 219 2414 64 \n",
       "Q 2138 -91 1772 -91 \n",
       "Q 1169 -91 856 284 \n",
       "Q 544 659 544 1381 \n",
       "z\n",
       "M 1991 3584 \n",
       "L 1991 3584 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-63\" d=\"M 3122 3366 \n",
       "L 3122 2828 \n",
       "Q 2878 2963 2633 3030 \n",
       "Q 2388 3097 2138 3097 \n",
       "Q 1578 3097 1268 2742 \n",
       "Q 959 2388 959 1747 \n",
       "Q 959 1106 1268 751 \n",
       "Q 1578 397 2138 397 \n",
       "Q 2388 397 2633 464 \n",
       "Q 2878 531 3122 666 \n",
       "L 3122 134 \n",
       "Q 2881 22 2623 -34 \n",
       "Q 2366 -91 2075 -91 \n",
       "Q 1284 -91 818 406 \n",
       "Q 353 903 353 1747 \n",
       "Q 353 2603 823 3093 \n",
       "Q 1294 3584 2113 3584 \n",
       "Q 2378 3584 2631 3529 \n",
       "Q 2884 3475 3122 3366 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "      </defs>\n",
       "      <use xlink:href=\"#DejaVuSans-23\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-20\" x=\"83.789062\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-74\" x=\"115.576172\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-6f\" x=\"154.785156\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-6b\" x=\"215.966797\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-65\" x=\"270.251953\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-6e\" x=\"331.775391\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-73\" x=\"395.154297\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-20\" x=\"447.253906\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-70\" x=\"479.041016\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-65\" x=\"542.517578\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-72\" x=\"604.041016\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-20\" x=\"645.154297\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-73\" x=\"676.941406\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-65\" x=\"729.041016\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-71\" x=\"790.564453\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-75\" x=\"854.041016\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-65\" x=\"917.419922\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-6e\" x=\"978.943359\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-63\" x=\"1042.322266\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-65\" x=\"1097.302734\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"matplotlib.axis_2\">\n",
       "    <g id=\"ytick_1\">\n",
       "     <g id=\"line2d_5\">\n",
       "      <defs>\n",
       "       <path id=\"m9adbfb286a\" d=\"M 0 0 \n",
       "L -3.5 0 \n",
       "\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use xlink:href=\"#m9adbfb286a\" x=\"66.053125\" y=\"143.1\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_6\">\n",
       "      <!-- 0 -->\n",
       "      <g transform=\"translate(52.690625 146.899219)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_2\">\n",
       "     <g id=\"line2d_6\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m9adbfb286a\" x=\"66.053125\" y=\"118.151116\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_7\">\n",
       "      <!-- 20000 -->\n",
       "      <g transform=\"translate(27.240625 121.950335)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-32\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-30\" x=\"63.623047\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-30\" x=\"127.246094\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-30\" x=\"190.869141\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-30\" x=\"254.492188\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_3\">\n",
       "     <g id=\"line2d_7\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m9adbfb286a\" x=\"66.053125\" y=\"93.202233\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_8\">\n",
       "      <!-- 40000 -->\n",
       "      <g transform=\"translate(27.240625 97.001451)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-34\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-30\" x=\"63.623047\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-30\" x=\"127.246094\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-30\" x=\"190.869141\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-30\" x=\"254.492188\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_4\">\n",
       "     <g id=\"line2d_8\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m9adbfb286a\" x=\"66.053125\" y=\"68.253349\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_9\">\n",
       "      <!-- 60000 -->\n",
       "      <g transform=\"translate(27.240625 72.052568)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-36\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-30\" x=\"63.623047\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-30\" x=\"127.246094\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-30\" x=\"190.869141\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-30\" x=\"254.492188\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_5\">\n",
       "     <g id=\"line2d_9\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m9adbfb286a\" x=\"66.053125\" y=\"43.304465\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_10\">\n",
       "      <!-- 80000 -->\n",
       "      <g transform=\"translate(27.240625 47.103684)scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-38\" d=\"M 2034 2216 \n",
       "Q 1584 2216 1326 1975 \n",
       "Q 1069 1734 1069 1313 \n",
       "Q 1069 891 1326 650 \n",
       "Q 1584 409 2034 409 \n",
       "Q 2484 409 2743 651 \n",
       "Q 3003 894 3003 1313 \n",
       "Q 3003 1734 2745 1975 \n",
       "Q 2488 2216 2034 2216 \n",
       "z\n",
       "M 1403 2484 \n",
       "Q 997 2584 770 2862 \n",
       "Q 544 3141 544 3541 \n",
       "Q 544 4100 942 4425 \n",
       "Q 1341 4750 2034 4750 \n",
       "Q 2731 4750 3128 4425 \n",
       "Q 3525 4100 3525 3541 \n",
       "Q 3525 3141 3298 2862 \n",
       "Q 3072 2584 2669 2484 \n",
       "Q 3125 2378 3379 2068 \n",
       "Q 3634 1759 3634 1313 \n",
       "Q 3634 634 3220 271 \n",
       "Q 2806 -91 2034 -91 \n",
       "Q 1263 -91 848 271 \n",
       "Q 434 634 434 1313 \n",
       "Q 434 1759 690 2068 \n",
       "Q 947 2378 1403 2484 \n",
       "z\n",
       "M 1172 3481 \n",
       "Q 1172 3119 1398 2916 \n",
       "Q 1625 2713 2034 2713 \n",
       "Q 2441 2713 2670 2916 \n",
       "Q 2900 3119 2900 3481 \n",
       "Q 2900 3844 2670 4047 \n",
       "Q 2441 4250 2034 4250 \n",
       "Q 1625 4250 1398 4047 \n",
       "Q 1172 3844 1172 3481 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-38\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-30\" x=\"63.623047\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-30\" x=\"127.246094\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-30\" x=\"190.869141\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-30\" x=\"254.492188\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_6\">\n",
       "     <g id=\"line2d_10\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m9adbfb286a\" x=\"66.053125\" y=\"18.355581\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_11\">\n",
       "      <!-- 100000 -->\n",
       "      <g transform=\"translate(20.878125 22.1548)scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-31\" d=\"M 794 531 \n",
       "L 1825 531 \n",
       "L 1825 4091 \n",
       "L 703 3866 \n",
       "L 703 4441 \n",
       "L 1819 4666 \n",
       "L 2450 4666 \n",
       "L 2450 531 \n",
       "L 3481 531 \n",
       "L 3481 0 \n",
       "L 794 0 \n",
       "L 794 531 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-31\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-30\" x=\"63.623047\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-30\" x=\"127.246094\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-30\" x=\"190.869141\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-30\" x=\"254.492188\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-30\" x=\"318.115234\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"text_12\">\n",
       "     <!-- count -->\n",
       "     <g transform=\"translate(14.798438 89.25625)rotate(-90)scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#DejaVuSans-63\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-6f\" x=\"54.980469\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-75\" x=\"116.162109\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-6e\" x=\"179.541016\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-74\" x=\"242.919922\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"patch_23\">\n",
       "    <path d=\"M 66.053125 143.1 \n",
       "L 66.053125 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_24\">\n",
       "    <path d=\"M 261.353125 143.1 \n",
       "L 261.353125 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_25\">\n",
       "    <path d=\"M 66.053125 143.1 \n",
       "L 261.353125 143.1 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_26\">\n",
       "    <path d=\"M 66.053125 7.2 \n",
       "L 261.353125 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"legend_1\">\n",
       "    <g id=\"patch_27\">\n",
       "     <path d=\"M 189.15 44.55625 \n",
       "L 254.353125 44.55625 \n",
       "Q 256.353125 44.55625 256.353125 42.55625 \n",
       "L 256.353125 14.2 \n",
       "Q 256.353125 12.2 254.353125 12.2 \n",
       "L 189.15 12.2 \n",
       "Q 187.15 12.2 187.15 14.2 \n",
       "L 187.15 42.55625 \n",
       "Q 187.15 44.55625 189.15 44.55625 \n",
       "z\n",
       "\" style=\"fill: #ffffff; opacity: 0.8; stroke: #cccccc; stroke-linejoin: miter\"/>\n",
       "    </g>\n",
       "    <g id=\"patch_28\">\n",
       "     <path d=\"M 191.15 23.798437 \n",
       "L 211.15 23.798437 \n",
       "L 211.15 16.798437 \n",
       "L 191.15 16.798437 \n",
       "z\n",
       "\" style=\"fill: #1f77b4\"/>\n",
       "    </g>\n",
       "    <g id=\"text_13\">\n",
       "     <!-- source -->\n",
       "     <g transform=\"translate(219.15 23.798437)scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#DejaVuSans-73\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-6f\" x=\"52.099609\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-75\" x=\"113.28125\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-72\" x=\"176.660156\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-63\" x=\"215.523438\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-65\" x=\"270.503906\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"patch_29\">\n",
       "     <path d=\"M 191.15 38.476562 \n",
       "L 211.15 38.476562 \n",
       "L 211.15 31.476562 \n",
       "L 191.15 31.476562 \n",
       "z\n",
       "\" style=\"fill: url(#h446a4b1fc9)\"/>\n",
       "    </g>\n",
       "    <g id=\"text_14\">\n",
       "     <!-- target -->\n",
       "     <g transform=\"translate(219.15 38.476562)scale(0.1 -0.1)\">\n",
       "      <defs>\n",
       "       <path id=\"DejaVuSans-61\" d=\"M 2194 1759 \n",
       "Q 1497 1759 1228 1600 \n",
       "Q 959 1441 959 1056 \n",
       "Q 959 750 1161 570 \n",
       "Q 1363 391 1709 391 \n",
       "Q 2188 391 2477 730 \n",
       "Q 2766 1069 2766 1631 \n",
       "L 2766 1759 \n",
       "L 2194 1759 \n",
       "z\n",
       "M 3341 1997 \n",
       "L 3341 0 \n",
       "L 2766 0 \n",
       "L 2766 531 \n",
       "Q 2569 213 2275 61 \n",
       "Q 1981 -91 1556 -91 \n",
       "Q 1019 -91 701 211 \n",
       "Q 384 513 384 1019 \n",
       "Q 384 1609 779 1909 \n",
       "Q 1175 2209 1959 2209 \n",
       "L 2766 2209 \n",
       "L 2766 2266 \n",
       "Q 2766 2663 2505 2880 \n",
       "Q 2244 3097 1772 3097 \n",
       "Q 1472 3097 1187 3025 \n",
       "Q 903 2953 641 2809 \n",
       "L 641 3341 \n",
       "Q 956 3463 1253 3523 \n",
       "Q 1550 3584 1831 3584 \n",
       "Q 2591 3584 2966 3190 \n",
       "Q 3341 2797 3341 1997 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-67\" d=\"M 2906 1791 \n",
       "Q 2906 2416 2648 2759 \n",
       "Q 2391 3103 1925 3103 \n",
       "Q 1463 3103 1205 2759 \n",
       "Q 947 2416 947 1791 \n",
       "Q 947 1169 1205 825 \n",
       "Q 1463 481 1925 481 \n",
       "Q 2391 481 2648 825 \n",
       "Q 2906 1169 2906 1791 \n",
       "z\n",
       "M 3481 434 \n",
       "Q 3481 -459 3084 -895 \n",
       "Q 2688 -1331 1869 -1331 \n",
       "Q 1566 -1331 1297 -1286 \n",
       "Q 1028 -1241 775 -1147 \n",
       "L 775 -588 \n",
       "Q 1028 -725 1275 -790 \n",
       "Q 1522 -856 1778 -856 \n",
       "Q 2344 -856 2625 -561 \n",
       "Q 2906 -266 2906 331 \n",
       "L 2906 616 \n",
       "Q 2728 306 2450 153 \n",
       "Q 2172 0 1784 0 \n",
       "Q 1141 0 747 490 \n",
       "Q 353 981 353 1791 \n",
       "Q 353 2603 747 3093 \n",
       "Q 1141 3584 1784 3584 \n",
       "Q 2172 3584 2450 3431 \n",
       "Q 2728 3278 2906 2969 \n",
       "L 2906 3500 \n",
       "L 3481 3500 \n",
       "L 3481 434 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "      </defs>\n",
       "      <use xlink:href=\"#DejaVuSans-74\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-61\" x=\"39.208984\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-72\" x=\"100.488281\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-67\" x=\"139.851562\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-65\" x=\"203.328125\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-74\" x=\"264.851562\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "  </g>\n",
       " </g>\n",
       " <defs>\n",
       "  <clipPath id=\"p4f217efe07\">\n",
       "   <rect x=\"66.053125\" y=\"7.2\" width=\"195.3\" height=\"135.9\"/>\n",
       "  </clipPath>\n",
       " </defs>\n",
       " <defs>\n",
       "  <pattern id=\"h446a4b1fc9\" patternUnits=\"userSpaceOnUse\" x=\"0\" y=\"0\" width=\"72\" height=\"72\">\n",
       "   <rect x=\"0\" y=\"0\" width=\"73\" height=\"73\" fill=\"#ff7f0e\"/>\n",
       "   <path d=\"M -36 36 \n",
       "L 36 -36 \n",
       "M -24 48 \n",
       "L 48 -24 \n",
       "M -12 60 \n",
       "L 60 -12 \n",
       "M 0 72 \n",
       "L 72 0 \n",
       "M 12 84 \n",
       "L 84 12 \n",
       "M 24 96 \n",
       "L 96 24 \n",
       "M 36 108 \n",
       "L 108 36 \n",
       "\" style=\"fill: #000000; stroke: #000000; stroke-width: 1.0; stroke-linecap: butt; stroke-linejoin: miter\"/>\n",
       "  </pattern>\n",
       " </defs>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<Figure size 252x180 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "#@save\n",
    "def show_list_len_pair_hist(legend, xlabel, ylabel, xlist, ylist):\n",
    "    \"\"\"绘制列表长度对的直方图\"\"\"\n",
    "    d2l.set_figsize()\n",
    "    _, _, patches = d2l.plt.hist(\n",
    "        [[len(l) for l in xlist], [len(l) for l in ylist]])\n",
    "    d2l.plt.xlabel(xlabel)\n",
    "    d2l.plt.ylabel(ylabel)\n",
    "    for patch in patches[1].patches:\n",
    "        patch.set_hatch('/')\n",
    "    d2l.plt.legend(legend)\n",
    "\n",
    "show_list_len_pair_hist(['source', 'target'], '# tokens per sequence',\n",
    "                        'count', source, target);"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "26e67b14",
   "metadata": {
    "origin_pos": 13
   },
   "source": [
    "## [**词表**]\n",
    "\n",
    "由于机器翻译数据集由语言对组成，\n",
    "因此我们可以分别为源语言和目标语言构建两个词表。\n",
    "使用单词级词元化时，词表大小将明显大于使用字符级词元化时的词表大小。\n",
    "为了缓解这一问题，这里我们将出现次数少于2次的低频率词元\n",
    "视为相同的未知（“&lt;unk&gt;”）词元。\n",
    "除此之外，我们还指定了额外的特定词元，\n",
    "例如在小批量时用于将序列填充到相同长度的填充词元（“&lt;pad&gt;”），\n",
    "以及序列的开始词元（“&lt;bos&gt;”）和结束词元（“&lt;eos&gt;”）。\n",
    "这些特殊词元在自然语言处理任务中比较常用。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "1179a522",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-08-18T07:07:37.732422Z",
     "iopub.status.busy": "2023-08-18T07:07:37.731864Z",
     "iopub.status.idle": "2023-08-18T07:07:37.957959Z",
     "shell.execute_reply": "2023-08-18T07:07:37.957157Z"
    },
    "origin_pos": 14,
    "tab": [
     "pytorch"
    ]
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "10012"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "src_vocab = d2l.Vocab(source, min_freq=2,\n",
    "                      reserved_tokens=['<pad>', '<bos>', '<eos>'])\n",
    "len(src_vocab)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "216c91b5",
   "metadata": {
    "origin_pos": 15
   },
   "source": [
    "## 加载数据集\n",
    ":label:`subsec_mt_data_loading`\n",
    "\n",
    "回想一下，语言模型中的[**序列样本都有一个固定的长度**]，\n",
    "无论这个样本是一个句子的一部分还是跨越了多个句子的一个片断。\n",
    "这个固定长度是由 :numref:`sec_language_model`中的\n",
    "`num_steps`（时间步数或词元数量）参数指定的。\n",
    "在机器翻译中，每个样本都是由源和目标组成的文本序列对，\n",
    "其中的每个文本序列可能具有不同的长度。\n",
    "\n",
    "为了提高计算效率，我们仍然可以通过*截断*（truncation）和\n",
    "*填充*（padding）方式实现一次只处理一个小批量的文本序列。\n",
    "假设同一个小批量中的每个序列都应该具有相同的长度`num_steps`，\n",
    "那么如果文本序列的词元数目少于`num_steps`时，\n",
    "我们将继续在其末尾添加特定的“&lt;pad&gt;”词元，\n",
    "直到其长度达到`num_steps`；\n",
    "反之，我们将截断文本序列时，只取其前`num_steps` 个词元，\n",
    "并且丢弃剩余的词元。这样，每个文本序列将具有相同的长度，\n",
    "以便以相同形状的小批量进行加载。\n",
    "\n",
    "如前所述，下面的`truncate_pad`函数将(**截断或填充文本序列**)。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "42aa524a",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-08-18T07:07:37.962377Z",
     "iopub.status.busy": "2023-08-18T07:07:37.961250Z",
     "iopub.status.idle": "2023-08-18T07:07:37.968643Z",
     "shell.execute_reply": "2023-08-18T07:07:37.967892Z"
    },
    "origin_pos": 16,
    "tab": [
     "pytorch"
    ]
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[47, 4, 1, 1, 1, 1, 1, 1, 1, 1]"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#@save\n",
    "def truncate_pad(line, num_steps, padding_token):\n",
    "    \"\"\"截断或填充文本序列\"\"\"\n",
    "    if len(line) > num_steps:\n",
    "        return line[:num_steps]  # 截断\n",
    "    return line + [padding_token] * (num_steps - len(line))  # 填充\n",
    "\n",
    "truncate_pad(src_vocab[source[0]], 10, src_vocab['<pad>'])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2974b660",
   "metadata": {
    "origin_pos": 17
   },
   "source": [
    "现在我们定义一个函数，可以将文本序列\n",
    "[**转换成小批量数据集用于训练**]。\n",
    "我们将特定的“&lt;eos&gt;”词元添加到所有序列的末尾，\n",
    "用于表示序列的结束。\n",
    "当模型通过一个词元接一个词元地生成序列进行预测时，\n",
    "生成的“&lt;eos&gt;”词元说明完成了序列输出工作。\n",
    "此外，我们还记录了每个文本序列的长度，\n",
    "统计长度时排除了填充词元，\n",
    "在稍后将要介绍的一些模型会需要这个长度信息。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "db17050b",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-08-18T07:07:37.973483Z",
     "iopub.status.busy": "2023-08-18T07:07:37.972873Z",
     "iopub.status.idle": "2023-08-18T07:07:37.978080Z",
     "shell.execute_reply": "2023-08-18T07:07:37.977330Z"
    },
    "origin_pos": 18,
    "tab": [
     "pytorch"
    ]
   },
   "outputs": [],
   "source": [
    "#@save\n",
    "def build_array_nmt(lines, vocab, num_steps):\n",
    "    \"\"\"将机器翻译的文本序列转换成小批量\"\"\"\n",
    "    lines = [vocab[l] for l in lines]\n",
    "    lines = [l + [vocab['<eos>']] for l in lines]\n",
    "    array = torch.tensor([truncate_pad(\n",
    "        l, num_steps, vocab['<pad>']) for l in lines])\n",
    "    valid_len = (array != vocab['<pad>']).type(torch.int32).sum(1)\n",
    "    return array, valid_len"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "85e2af67",
   "metadata": {
    "origin_pos": 19
   },
   "source": [
    "## [**训练模型**]\n",
    "\n",
    "最后，我们定义`load_data_nmt`函数来返回数据迭代器，\n",
    "以及源语言和目标语言的两种词表。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "8addcc51",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-08-18T07:07:37.982873Z",
     "iopub.status.busy": "2023-08-18T07:07:37.982349Z",
     "iopub.status.idle": "2023-08-18T07:07:37.988101Z",
     "shell.execute_reply": "2023-08-18T07:07:37.987357Z"
    },
    "origin_pos": 20,
    "tab": [
     "pytorch"
    ]
   },
   "outputs": [],
   "source": [
    "#@save\n",
    "def load_data_nmt(batch_size, num_steps, num_examples=600):\n",
    "    \"\"\"返回翻译数据集的迭代器和词表\"\"\"\n",
    "    text = preprocess_nmt(read_data_nmt())\n",
    "    source, target = tokenize_nmt(text, num_examples)\n",
    "    src_vocab = d2l.Vocab(source, min_freq=2,\n",
    "                          reserved_tokens=['<pad>', '<bos>', '<eos>'])\n",
    "    tgt_vocab = d2l.Vocab(target, min_freq=2,\n",
    "                          reserved_tokens=['<pad>', '<bos>', '<eos>'])\n",
    "    src_array, src_valid_len = build_array_nmt(source, src_vocab, num_steps)\n",
    "    tgt_array, tgt_valid_len = build_array_nmt(target, tgt_vocab, num_steps)\n",
    "    data_arrays = (src_array, src_valid_len, tgt_array, tgt_valid_len)\n",
    "    data_iter = d2l.load_array(data_arrays, batch_size)\n",
    "    return data_iter, src_vocab, tgt_vocab"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6afba4ea",
   "metadata": {
    "origin_pos": 21
   },
   "source": [
    "下面我们[**读出“英语－法语”数据集中的第一个小批量数据**]。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "90df834d",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-08-18T07:07:37.992732Z",
     "iopub.status.busy": "2023-08-18T07:07:37.992204Z",
     "iopub.status.idle": "2023-08-18T07:07:43.780428Z",
     "shell.execute_reply": "2023-08-18T07:07:43.779613Z"
    },
    "origin_pos": 22,
    "tab": [
     "pytorch"
    ]
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "X: tensor([[ 7, 43,  4,  3,  1,  1,  1,  1],\n",
      "        [44, 23,  4,  3,  1,  1,  1,  1]], dtype=torch.int32)\n",
      "X的有效长度: tensor([4, 4])\n",
      "Y: tensor([[ 6,  7, 40,  4,  3,  1,  1,  1],\n",
      "        [ 0,  5,  3,  1,  1,  1,  1,  1]], dtype=torch.int32)\n",
      "Y的有效长度: tensor([5, 3])\n"
     ]
    }
   ],
   "source": [
    "train_iter, src_vocab, tgt_vocab = load_data_nmt(batch_size=2, num_steps=8)\n",
    "for X, X_valid_len, Y, Y_valid_len in train_iter:\n",
    "    print('X:', X.type(torch.int32))\n",
    "    print('X的有效长度:', X_valid_len)\n",
    "    print('Y:', Y.type(torch.int32))\n",
    "    print('Y的有效长度:', Y_valid_len)\n",
    "    break"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "df773107",
   "metadata": {
    "origin_pos": 23
   },
   "source": [
    "## 小结\n",
    "\n",
    "* 机器翻译指的是将文本序列从一种语言自动翻译成另一种语言。\n",
    "* 使用单词级词元化时的词表大小，将明显大于使用字符级词元化时的词表大小。为了缓解这一问题，我们可以将低频词元视为相同的未知词元。\n",
    "* 通过截断和填充文本序列，可以保证所有的文本序列都具有相同的长度，以便以小批量的方式加载。\n",
    "\n",
    "## 练习\n",
    "\n",
    "1. 在`load_data_nmt`函数中尝试不同的`num_examples`参数值。这对源语言和目标语言的词表大小有何影响？\n",
    "1. 某些语言（例如中文和日语）的文本没有单词边界指示符（例如空格）。对于这种情况，单词级词元化仍然是个好主意吗？为什么？\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "439397fa",
   "metadata": {
    "origin_pos": 25,
    "tab": [
     "pytorch"
    ]
   },
   "source": [
    "[Discussions](https://discuss.d2l.ai/t/2776)\n"
   ]
  }
 ],
 "metadata": {
  "language_info": {
   "name": "python"
  },
  "required_libs": []
 },
 "nbformat": 4,
 "nbformat_minor": 5
}